Image determining device to determine the state of a subject

ABSTRACT

To determine the state of a subject person with a simple structure, an image determining device includes: an imaging unit that captures an image from a first direction, the image including the subject person; a first detector that detects size information from the image, the size information being about the subject person in the first direction; a second detector that detects position-related information, the position-related information being different from the information detected by the first detector; and a determining unit that determines the state of the subject person, based on a result of the detection performed by the first detector and a result of the detection performed by the second detector.

This Application is a Divisional Application of application Ser. No.14/700,960 filed Apr. 30, 2015, which is a Divisional of applicationSer. No. 13/513,704 filed Jun. 4, 2012, which is a National PhaseApplication of International Application No. PCT/JP2010/073748 filedDec. 28, 2010, which claims the benefit of Japanese Application No.2010-001903 filed Jan. 7, 2010, Japanese Application No. 2010-001902filed Jan. 7, 2010 and Japanese Application No. 2010-001901 filed Jan.7, 2010. The disclosure of each of the prior applications isincorporated herein by reference in its entirety.

TECHNICAL FIELD

The present invention relates to a security system using an imagedetermining device.

BACKGROUND ART

There have been video monitoring devices that were developed forcapturing images of elderly people living alone with a camera anddetermining presence or absence of a problem. Also, to protect theprivacy of a person to be imaged by a camera, adding a mosaic pattern toeach captured video image has been suggested (Patent Document 1, forexample).

PRIOR ART DOCUMENT Patent Document

Patent Document 1: Japanese Unexamined Patent Application PublicationNo. 2002-142214

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

However, a conventional video monitoring device constantly capturesimages with a camera to detect a problem in a subject person, andmonitors the subject person with another monitoring device. Therefore,complicated processes need to be performed.

Means for Solving the Problems

An image determining device of the present invention comprises: animaging unit that captures an image from a first direction, the imageincluding a subject person; a first detector that detects sizeinformation from the image, the size information being about the subjectperson in the first direction; a second detector that detectsposition-related information, the position-related information beingdifferent from the information detected by the first detector; and adetermining unit that determines a state of the subject person, based ona result of the detection performed by the first detector and a resultof the detection performed by the second detector.

In this case, the second detector may detect position information withrespect to a second direction, the second direction being different fromthe first direction. In this case, the second detector may detectposition information about the subject person or a location near thesubject person in the second direction. Also, the first detector maydetect a size of the head of the subject person, and the determiningunit may determine the state of the subject person by associating thesecond-direction position information with the size of the head of thesubject person. Further, the image determining device may furthercomprise a storage unit that stores a reference value for the sizeinformation, in accordance with the second-direction positioninformation.

Also, the image determining device of the present invention may beconfigured so that the first detector detects a size of the head of thesubject person. Also, the determining unit may determine the state ofthe subject person, based on a plurality of images captured at differenttimes. Also, the imaging unit may image the subject person at firstpredetermined intervals, and, in accordance with a result of thedetermination made by the determining unit, may image the subject personat second predetermined intervals, the second predetermined intervalsbeing shorter than the first predetermined intervals. Further, theimaging unit may capture a still image.

The image determining device of the present invention may furthercomprise a determining sensor that determines whether the subject personexists. In this case, the imaging unit may perform the imaging inaccordance with a result of the determination made by the determiningsensor.

The image determining device of the present invention may furthercomprises a communicating unit that communicates with another device. Inthis case, when the communicating unit sends a result of thedetermination on the subject person to the another device, thedetermining unit may prohibit at least sending of the image captured atfirst predetermined intervals. Also, in this case, the determining unitmay determine whether to allow sending of the image captured at secondpredetermined intervals, the second predetermined intervals beingshorter than the first predetermined intervals.

Also, the image determining device of the present invention may furthercomprise a second power source that is different from a first powersource supplying power to the imaging unit.

Also, the image determining device of the present invention may beconfigured so that the second detector is a relative position detectorthat detects information about a relative position between the imagingunit and the subject person. In this case, the imaging unit may includean optical system, and the relative position detector detectsinformation about a relative position between the optical system and thesubject person. Also, the relative position detector may detect ananticipated angle of the optical system with respect to the subjectperson. Further, the imaging unit may include an imaging element, andthe relative position detector may detect the information about therelative position between the imaging unit and the subject person, basedon a position of an image of the subject person captured by the imagingelement.

The image determining device of the present invention may furthercomprise a correcting unit that corrects the size information about thesubject person detected by the first detector, in accordance with aresult of the detection performed by the relative position detector.Also, the image determining may be configured so that the imaging unitimages a structural object and the subject person, and based on theimaged structural object and the imaged subject person, the determiningunit determines the state of the subject person.

An image determining device of the present invention comprises: animaging unit that captures an image from a first direction; an acquiringunit that acquires height information about a first subject person andheight information about a second subject person; and an identifyingunit that identifies the first subject person and the second subjectperson in accordance with the height information from the acquiringunit, when the captured image includes at least one of the first subjectperson and the second subject person.

In this case, the identifying unit may determine a state of at least oneof the first subject person and the second subject person, in accordancewith the height information from the acquiring unit. Also, the imagingunit may image the heads of the first subject person and the secondsubject person. Also, the imaging unit may perform imaging when at leastone of the first subject person and the second subject person moves froman imaging area of the imaging unit to a non-imaging area.

The image determining device may be configured so that the acquiringunit acquires respective heights of the first subject person and thesecond subject person as the height information. Also, the acquiringunit may acquire the height information in accordance with a result ofthe imaging performed on the first subject person and the second subjectperson by the imaging unit. Also, the image determining device may beconfigured so that the imaging unit images a structural object, andbased on the imaged structural object, the acquiring unit acquires theheight information about at least one of the first subject person andthe second subject person.

The image determining device of the present invention may be configuredso that the imaging unit simultaneously images the first subject personand the second subject person. Further, the image determining device maybe configured so that the imaging unit images the first subject personand the second subject person when the first subject person and thesecond subject person are in the same posture.

Effects of the Invention

According to the present invention, the privacy of a subject person canbe secured, and the state of the subject person can be determined with asimple structure.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of an image determining device 1 according tothis embodiment;

FIG. 2 is a diagram showing the main parts of the image determiningdevice 1 of this embodiment;

FIG. 3 is a diagram showing the structure of the imaging unit 2;

FIG. 4 is a diagram showing changes in the size of the head inaccordance with postures of the subject person;

FIGS. 5A and 5B are graphs showing changes in the size of an imageformed by the wide-angle lens system; wherein FIG. 5A is a graph showingthe distance from the front-side focal point of the wide-angle lenssystem to the head of the subject person, and the image size; and FIG.5B is a graph formed by converting FIG. 5A into the height from thefloor;

FIG. 6 is a graph that shows the rate of change in the image size basedon FIGS. 5A and 5B;

FIG. 7 is a flowchart about a case where the image determining device isinstalled in a living room;

FIG. 8 is a diagram illustrating a case where the subject person is in awalking area in the living room;

FIG. 9 is a diagram illustrating a case where the subject person is nearthe closet;

FIG. 10 is a diagram showing a change in the image size of the head ofthe subject person imaged in the imaging element in accordance withpositions of the subject person;

FIG. 11 is a flowchart showing operations of the image determiningdevice;

FIG. 12 is a diagram showing the influence of distortions of thewide-angle lens system in accordance with anticipated angles withrespect to the subject person;

FIG. 13 is a diagram illustrating a situation where the subject personis sitting in a chair in front of a desk; and

FIG. 14 is a still image captured when the subject person is determinedto be in an abnormal state.

MODES FOR CARRYING OUT THE INVENTION First Embodiment

An image determining device of this embodiment can be placed in a roomof a house or office or the like, and a case where the image determiningdevice is placed in a living room is described in the following.

FIG. 1 is a block diagram of the image determining device 1 of thisembodiment. FIG. 2 is a diagram showing the main parts of the imagedetermining device 1. As shown in FIGS. 1 and 2, the image determiningdevice 1 includes: an imaging unit 2 that performs imaging; acommunicating unit 3 that communicates with another device; anilluminometer 4 that measures the illuminance in the living room whenthe imaging unit 2 performs imaging; a LED (Light Emitting Diode) 5 thatilluminates a subject person in the living room when the illuminancemeasured by the illuminometer 4 is low; a pyroelectric sensor 6 thatdetermines whether the subject person is present; a setting unit 7 thatconfigures various kinds of settings in the image determining device 1;an interface unit 8 for cooperating with a lighting fixture or anelectric appliance in the living room; a microphone 9 that records thevoice of the subject person; and a speaker 10 for speaking to thesubject person. The image determining device 1 also includes: a flashmemory 11 that is a storage medium storing captured images and positioninformation as to a specific point in the living room when the subjectperson is present in the living room; and a calendar unit 12 thatincludes a crystal oscillator and a timer integrated circuit,automatically updates calendar information indicating the year, month,day, hour, minute, and second, and stores plans of the subject person.

The image determining device 1 is also connected to a lithium-ion cell13 as a backup power source in case power from the primary source (ahousehold power source in this embodiment) is not supplied due to apower failure or the like. The backup power source is not necessarily alithium-ion cell. A household solar power system may be used, or acapacitor may be used in place of the lithium-ion battery.

A CPU 14 controls the entire image determining device 1, and iselectrically connected to the respective components. It goes withoutsaying that the CPU 14 and the setting unit 7 are electrically connectedto each other even if the setting unit 7 is a remote controller.

(Imaging Unit 2)

The imaging unit 2 is placed in the center of the ceiling of the livingroom, and mainly captures images of the head of the subject person. Inthis embodiment, the height of the ceiling of the living room is 2.6 m(see FIG. 4).

FIG. 3 is a diagram showing the structure of the imaging unit 2.

As shown in FIG. 3, the imaging unit 2 includes: a wide-angle lenssystem with a three-group structure; a low-pass filter; an imagingelement formed with a CCD or CMOS or the like; and a circuit board thatdrives and controls the imaging element. The imaging element of thisembodiment is 23.7 mm×15.9 mm in size, and has 4000×3000 (12 millions)pixels, with the size of each one pixel being 5.3 μm. However, the sizeand number are not limited to the above.

The above wide-angle lens system includes: a first group including twonegative meniscus lenses; a second group including a positive lens, acemented lens, and an infrared cutoff filter; and a third groupincluding two cemented lenses. A diaphragm is provided between thesecond group and the third group. The wide-angle lens system of thisembodiment has a focal length of 6.188 mm in the entire system, and hasa maximum view angle of 80 degrees.

The wide-angle lens system is not limited to the above three-groupstructure, and it is possible to arbitrarily change the number of lensesin each group, the lens structure, the focal length, and the angle ofview.

FIGS. 5A and 5B are graphs showing changes in the size of an imageformed by the wide-angle lens system. FIG. 5A is a graph showing thedistance from the front-side focal point of the wide-angle lens systemto the head of the subject person, and the image size. FIG. 5B is agraph formed by converting FIG. 5A into the height from the floor.

Where the wide-angle lens system of this embodiment has a focal lengthof 6.188 mm as described above, and the diameter of the head of thesubject person is 200 mm, the size of the head of the subject personimaged in the imaging element of the imaging unit 2 is 1.238 mm when thedistance from the front-side focal point of the wide-angle lens systemto the position of the head of the subject person is 1000 mm (when thesubject person is standing). When the head of the subject person islowered from that position by 300 mm, and the distance from thefront-side focal point of the wide-angle lens system to the position ofthe head of the subject person becomes 1300 mm, the size of the head ofthe subject person imaged in the imaging element of the imaging unit 2becomes 0.952 mm, and there is a 0.286 mm (23.1%) change in the imagesize.

Likewise, when the distance from the front-side focal point of thewide-angle lens system to the position of the head of the subject personis 2000 mm (when the subject person is in a semi-crouching position),the size of the head of the subject person imaged in the imaging elementof the imaging unit 2 is 0.619 mm. When the head of the subject personis lowered from that position by 300 mm, the size of the head of thesubject person imaged in the imaging element of the imaging unit 2becomes 0.538 mm, and there is a 0.081 mm (13.1%) change in the imagesize. From that, it is apparent that as the distance from the front-sidefocal point of the wide-angle lens system to the head of the subjectperson becomes longer, the change in the image size becomes smaller.

As described above, with the image determining device 1 of thisembodiment, the distance from the front-side focal point of thewide-angle lens system to the subject person is determined from the sizeof an image of the head of the subject person when the size of the headof the subject person is known. Accordingly, it is possible to determinethe posture of the subject person (standing, semi-crouching, or lying onthe floor) and a change in the posture while maintaining the privacy ofthe subject person.

FIG. 6 is a graph that shows the rate of change in the image size basedon FIGS. 5A and 5B, and shows the rate of change in the size of theimage formed by the wide-angle lens system every time the position ofthe head of the subject person is changed by 100 mm.

When the distance from the front-side focal point of the wide-angle lenssystem to the position of the head of the subject person is increasedfrom 1000 mm by 100 mm, the rate of change in the image size is as highas 9.1%. Therefore, with the image determining device 1 of thisembodiment, two or more subject persons can be identified from thedifference in height, if the height difference is at least approximately100 mm.

When the distance from the front-side focal point of the wide-angle lenssystem to the position of the head of the subject person is increasedfrom 2000 mm by 100 mm, the rate of change in the image size becomes aslow as 4.8%. With the image determining device 1 of this embodiment,however, the change in the posture of the subject person can be detectedeven if the distance from the front-side focal point of the wide-anglelens system to the position of the head of the subject person becomeslonger, which will be described later in detail.

FIGS. 5A, 5B and 6 are graphs showing cases where the subject person ispresent in a position at a low angle of view of the wide-angle lenssystem (immediately below the wide-angle lens system). If the subjectperson is present in a peripheral view-angle position of the wide-anglelens system, there would be the influence of a distortion depending onthe anticipated angle with respect to the subject person, which is notshown in the graphs of FIGS. 5A, 5B and 6. The influence of a distortionwill be described later.

Luminous flux incident on the wide-angle lens system enters the imagingelement via the low-pass filter, and the circuit board converts theoutput of the imaging element into a digital signal. An image processingcontrol unit formed with an ASIC performs image processing, such aswhite balance adjustment, sharpness adjustment, gamma correction, andgray level correction, on the image signal converted into the digitalsignal, and also performs image compression using JPEG or the like. Thestill image compressed using JPEG is recorded on the flash memory 11. Torecord a video image, the image signal should be subjected to MPEGprocessing, and should be recorded on the flash memory 11.

FIG. 4 is a diagram showing changes in the size of the head inaccordance with postures of the subject person. As shown in FIG. 4, theimaging unit 2 is attached to the ceiling of the living room, andcaptures an image of the position of the head of the subject person.When the subject person is standing (the left side), the size of thehead in the image is large. When the subject person is lying on thefloor (the right side), the size of the head in the image is small. Whenthe subject person is in a semi-crouching position (the center), thesize of the head in the image is smaller than that imaged when thesubject person is standing, but is larger than that imaged when thesubject person is lying on the floor. In this embodiment, a check ismade to determine whether the subject person is fine, based on the stateof the subject person or the size of the head that varies in theZ-direction. In this manner, attention is paid to the size of the headof the subject person that varies in the Z-direction as described above.Accordingly, the privacy of the subject person can be protected.

When there is something wrong with the subject person and the subjectperson is lying on the floor or the like, a zoom lens added to the abovedescribed wide-angle lens system continuously captures images of the eyeand mouth, so that a check can be made to determine whether the eye isopen based on pattern matching, whether the eyeball is moving, whetherthe mouth is moving, or whether the subject person is breathing based onmovement of the mouth.

Further, the high-frequency component is extracted from the signalsupplied from the imaging element, and the lens position in which thehigh-frequency component become largest is detected. In this manner,focus detection is performed, and this is known as contrast AF. ContrastAF is also applied to this embodiment, and part of the wide-angle lenssystem may be adjusted to an optimum position.

Normally, the imaging by the imaging unit 2 only involves capturing 48still images per day if the imaging is performed once every 30 minutes.Accordingly, the power supplied to the image determining device 1 andthe recording area in the flash memory 11 can be saved. Also, inaccordance with date information supplied from the calendar unit 12, theimaging interval may be varied with the time of day. Specifically, theimaging interval may be 30 minutes in the daytime, and may be 20 minutesin the nighttime. If the imaging element is a CMOS, and an electronicshutter (a rolling shutter) is used, instead of a mechanical shutter, nonoise would be generated at the time of imaging, and the sleep of thesubject person would not be interrupted.

Also, instead of still images, short video images each lasting 3 to 5seconds may be captured.

When the CPU 14 detects a problem from a captured image, the imagingunit 2 captures about a hundred successive images per minute, orcaptures about a hundred images at 15-second intervals, in accordancewith an instruction from the CPU 14. The imaging interval may be set byusing the setting unit 7.

The imaging time and ISO sensitivity at the time when the imaging unit 2captures still images are determined by the illuminance detected by theilluminometer 4. When the living room is dark, the ISO sensitivityshould be made higher, and the imaging time should be prolonged. In acase where the image determining device 1 is placed in a bedroom, andthe subject person puts out all the lights when asleep, the LED 5 mayilluminate the subject person, or the interface unit 8 cooperates withthe lighting fixture in the bedroom, to illuminate the subject personfor a few seconds (approximately 3 seconds) and secure the brightnessnecessary for imaging. The pyroelectric sensor 6 checks the position ofthe subject person, and the position of the LED 5 is adjusted by a driveunit (not shown) so as to illuminate the checked position withilluminating light. In this manner, the subject person can beefficiently illuminated. The LED 5 may be attached to the ceiling, ormay be placed at any location in the bedroom. A light source other thana LED may be used. If the lighting fixture in the bedroom suffices, theLED 5 may not be provided. As the lighting fixture in the bedroom, a LEDlamp may be used.

Further, captured still images of the subject person are only requiredto be clear enough to detect a problem of the subject person, andtherefore, those still images differ from conventional still images forviewing. When imaging is performed in a dark-lit room, the infraredcutoff filter provided in the wide-angle lens system may be retracted bya retracting mechanism (not shown), so as to secure a sufficient amountof light.

(Communicating Unit 3)

The communicating unit 3 is formed with an antenna, a transmitter, areceiver, and the like. In this embodiment, the communicating unit 3 canbe placed in a corner of a living room. The communicating unit 3 reportsthe occurrence or non-occurrence of a problem of the subject person to adesignated other device (a portable telephone of a member of the familyof the subject person or a personal computer at a security company), andreceives audio information and the like from the other device. Whenthere is nothing wrong with the subject person, the communicating unit 3transmits a message indicating no problem, together with dateinformation, to the designated other device, but does not transmit anycaptured image. In this manner, the privacy of the subject person can beprotected. As will be described later, when there is something wrongwith the subject person, images can be transmitted under predeterminedconditions.

(Pyroelectric Sensor 6)

The pyroelectric sensor 6 is an infrared sensor. The pyroelectric sensor6 detects infrared rays generated from a person, and outputs thedetection result to the CPU 14. In this embodiment, the pyroelectricsensor 6 is placed so as to detect infrared rays via the wide-angle lenssystem. Pyroelectric sensors 6 may be provided in the living room.

Upon detecting the subject person entering the living room based on theoutput of the pyroelectric sensor 6, the CPU 14 instructs the imagingunit 2 to perform imaging, and may cause the imaging unit 2 to performimaging thereafter at 30-minute intervals. If the subject person movesout of the living room and returns a few minutes later, there is no needto reset the time counter.

Upon detecting absence of the subject person from the living room basedon the output of the pyroelectric sensor 6, the CPU 14 controls theimaging unit 2 not to perform imaging even at a scheduled time forimaging. In a case where the subject person does not enter the livingroom (and is not present in any other room) for a few days, and there isnot a reported plan of absence from home in the schedule stored in thecalendar unit 12, the CPU 14 reports the absence of the subject personand a possibility of a problem to the designated other device via thecommunicating unit 3.

(Calendar Unit 12)

As described above, the calendar unit 12 automatically updates calendarinformation indicating the year, month, day, hour, minute, and second,and stores plans of the subject person. A plan of the subject person maybe input through the setting unit 7, or a date (duration) anddestination may be input through the microphone 9. In that case, thecalendar unit 12 is equipped with a voice recognition function. However,only a simple phonetic dictionary for dates and destinations and thelike needs to be prepared, and accordingly, the system does not becomelarger. A plan may be voice-input through the microphone 9, in responseto an inquiry (as to when and where) from the speaker 10.

The calendar unit 12 notifies the subject person of the next day's planvia the speaker 10 at night, and again notifies the next morning.

In accordance with a plan that is input to the calendar unit 12, the CPU14 may set an alarm clock or display the plan on a television screen viathe interface unit 8. Also, where a request to watch a televisionprogram is input, the CPU 14 may cause the television set to be beautomatically turned on or cause the requested program to be recorded.

(Calibration)

FIG. 7 is a flowchart about a case where the image determining device ofthis embodiment is installed in a living room. In the following, theinstallation of the image determining device is described, withreference to the flowchart of FIG. 7.

In step 1, a communication device to which presence or absence of aproblem of the subject person is to be reported is designated. Here, amobile device such as a portable telephone or a personal computer of afamily member, a personal computer of a security company, or the likecan be designated. Also, privacy levels of the family member and asecurity company can be set. For example, in a case where the privacylevel of the family member may be lowered, transfers of images capturedwhen a problem occurs are allowed, or the later described maskprocessing can be cancelled or can be given a lower mask processinglevel. In a case where the privacy level of a security company is madehigher than that of a family member, transfers of images captured when aproblem occurs can be prohibited, and only transfers of images maskedthrough the later described mask processing can be allowed.

Based on the designation made in step 1, the CPU 14 reports presence orabsence of a problem of the subject person to the designated device viathe communicating unit 3.

In step 2, a connection device such as a television set, an audiodevice, or a clock is designated. Specifically, a wired or wirelessconnection with a television set is made via the interface unit 8, forexample, so that the activity schedule stored in the calendar unit 12 isdisplayed on the television set, the television set is turned on a dateand a time designated in the calendar unit 12, or a requested program isrecorded. Likewise, an audio device is turned on or an alarm clock isstarted on a date and a time stored in the calendar unit 12.

In step 3, the CPU 14 stores the relationship between a typical positionin the living room and the position of the head of the subject personinto the flash memory 11.

FIG. 8 is a diagram illustrating a case where the subject person is in awalking area in the living room (in an area where sofas, tables, chairs,and the like are not placed in the X-direction and the Y-direction).

The CPU 14 captures an image of the head of the subject person with theimaging unit 2 when the subject person is standing in the living room asshown in FIG. 8, for example. The CPU 14 associates the position of thehead in the Z-direction with the size of the head in the Z-directionwhen the subject person is standing, and stores the relationship intothe flash memory 11. At this point, the CPU 14 also recognizes aposition in the X-direction and the Y-direction in which the subjectperson is likely to stand in the living room. Specifically, the CPU 14recognizes a position in which sofas, tables, chairs, and the like arenot placed. Further, in addition to the size of the head of the subjectperson in the Z-direction, the CPU 14 may estimate, from the capturedimage, the size of the head in a region between a position Z1 slightlyhigher than the position of the head of the subject person and aposition Z2 slightly lower than the position of the head of the subjectperson. The CPU 14 may store the estimated size of the head. In thismanner, the allowable range of the size of the head when the subjectperson is in a normal state can be set.

FIG. 9 is a diagram illustrating a case where the subject person is nearthe closet.

When the subject person is near the closet, the subject person isconsidered to be standing or in a crouching position. Therefore, the CPU14 captures an image of the head of the subject person in a crouchingposition with the imaging unit 2, as shown in FIG. 9. The CPU 14 thenassociates the position with the size at a height Z3, and stores therelationship into the flash memory 11. The CPU 14 also sets sizes of thehead in a region between the heights Z1 and Z2 as the allowable range inan area near the closet. When the size of the head of the subject personis equal to or smaller than the size at the height Z3 over a long periodof time even though the size is within the allowable range between theheights Z1 and Z2, there is a possibility that a problem has occurred inthe subject person. In such a case, the CPU 14 continuously capturesimages of the head. Based on pattern matching, the CPU 14 determinesthat the subject person is in a normal state when the head is moving,but talks to the subject person through the speaker 10 when the head isnot moving.

FIG. 10 is a diagram showing a change in the size of the head of thesubject person imaged in the imaging element in accordance withpositions of the subject person. Here, the center of the imaging elementmatches the center of the optical axis of the wide-angle lens system.

Even in a case where the subject person is standing in the living roomas shown in FIG. 8, the size of the head imaged by the imaging unit 2varies between when the subject person is standing near the door andwhen the subject person is standing immediately below the imaging unit2.

As shown in FIG. 10, the image size, the distance L1 from the center ofthe imaging element, and the angle θ1 with respect to the center of theimaging element are determined from an image formed at P1 in the imagingelement. The distance L1 from the center of the imaging element is aparameter representing the distance between the front-side focal pointof the wide-angle lens system and the head of the subject person. Theangle θ1 with respect to the center of the imaging element is aparameter representing the anticipated angle of the wide-angle lenssystem with respect to the subject person. As described above, evenwhere the subject person is in the same posture, the size of the imageformed at P1 in the imaging element and the size of the image formed atP2 in the imaging element are different from each other. Therefore, theCPU 14 corrects the sizes of the captured images, based on the distancesL1 and L2 from the center of the imaging element, and the angles θ1 andθ2 with respect to the center of the imaging element. In other words,where the subject person is in the same posture, a correction isperformed so that the size of the image formed at P1 in the imagingelement and the size of the image formed at P2 in the imaging elementbecome substantially equal to each other. The parameters related to thecorrection are also stored into the flash memory 11. In the abovedescription, the position of the subject person is detected based on theposition of an image formed in the imaging element. Instead, thedistance between the imaging unit 2 and the subject person may bedetected by emitting light toward the subject person and receiving thelight reflected from the subject person.

As described above, the image determining device 1 of this embodimentcan detect the positional relationship between the imaging unit 2 andthe subject person (the distance to the subject person and theanticipated angle with respect to the subject person). Accordingly, theposture of the subject person can be accurately detected, no matterwhich part of the living room the subject person is in.

In a case where there are two or more subject persons, the heights andthe head sizes vary. Therefore, head sizes of each subject person inaccordance with positions of the head in the Z-direction, and theposition information in the X- and Y-directions are stored into theflash memory 11. In that case, images of the subject persons may besimultaneously captured by the imaging unit 2, or may be capturedseparately from one another.

Further, the heights and the head sizes (the diameters of the heads) ofthe subject persons may be input through the setting unit 7, and bestored into the flash memory 11. When a first subject person enters theliving room through the door, and the head of the first subject personis imaged by the imaging unit 2, for example, the CPU 14 can recognizethe first subject person entering the living room based on the size ofthe captured image and the height information stored in the flash memory11, as the heights and the head sizes of the subject persons are storedin the flash memory 11. In this manner, the heights of two or moresubject persons are stored into the flash memory 11, so that the subjectpersons can be identified based on the height differences even thoughthere are no differences in the head size, while the privacy of thesubject persons is maintained.

However, when the subject persons are in semi-crouching positions or arelying on the floor, there are no substantial differences in height, andthe subject persons cannot be identified. Even in such a situation, oneof particularly serious problems occurs when one of the subject personsis left in the living room.

If images of subject persons leaving the living room are captured nearthe door, the CPU 14 can determine which ones have left the living roomand which one remains in the living room, using the output of thepyroelectric sensor 6. In this manner, the imaging unit 2 of thisembodiment captures images of subject persons before the subject personsmove out of the imaging area of the imaging unit 2, so that not onlypresence or absence of a person in the living room but also which oneremains in the living room can be determined.

Also, a template image of a subject person (an image of the head of thesubject person captured by the imaging unit 2) may be stored into theflash memory 11, and template matching may be performed, to identify thesubject person who is lying on the floor.

When a subject person enters the living room, an image of the subjectperson standing immediately below the imaging unit 2 is captured, sothat the subject person can be accurately identified based on theheight.

Instead of inputting the height and the head size of a subject person,images of a structural object (such as the door) in the living room andthe subject person may be captured, and, if the size of the structuralobject (the height of the door being 2 m, the width being 1 m, forexample) is known, the height and the head size of the object may beestimated from the structural object. In this case, the size of thestructural object should be recorded on the flash memory 11, and theimaging unit 2 preferably captures an image of each person in the sameposture (a standing posture).

In a case where a subject person is a child, the height and the headsize increase. Therefore, a reference still image of the child should betaken once a year, for example. When the layout of the room is changed,the relationship between head sizes corresponding to positions of thehead of each subject person in the Z-direction and the positioninformation in the X- and Y-directions should be updated.

Further, based on the output of the pyroelectric sensor 6, the CPU 14stores the times of the day at which the subject person spends time inthe living room on weekdays and holidays are stored into the flashmemory 1.

In step 4, masking is performed on the captured image. If the subjectperson does not request the masking, this step may be skipped. Themasking is performed by lowering the resolution of the entire capturedimage, or adding a mosaic pattern to the parts other than the head orlowering the resolution of the parts other than the head, or detectingthe flesh-colored portions from the captured image and adding a mosaicpattern to the flesh-colored portions or lowering the resolution of theflesh-colored portions, for example.

In step 5, the settings, such as the time intervals at which an image iscaptured, and the volume of sound from the speaker 10, are checked. Whensound is emitted from the speaker 10 or the voice of the subject personis collected by the microphone 9, the CPU 14 controls the interface unit8 to lower the volume of sound emitted from the television set and theaudio device.

(Operation Flowchart)

FIG. 11 is a flowchart showing operations of the image determiningdevice of this embodiment. In the following, the explanation iscontinued with reference to this flowchart.

Based on the output of the pyroelectric sensor 6, the CPU 14 determineswhether the subject person is present in the living room (step 1). Ifthe subject person is present in the living room, the CPU 14 moves on tostep 2. If the subject person is not present in the living room, the CPU14 repeats step 1.

The CPU 14 controls the calendar unit 12, to start the timing (step 2).Step 2 is the step for setting the intervals at which a still image iscaptured when the subject person is in a normal state, in conjunctionwith step 8, which will be described later.

The CPU 14 determines whether the number of times a still image has beencaptured is a predetermined number of times (step 3). Step 3 is the stepfor taking measures to plot the next move when the state of the subjectperson cannot be determined even after continuous still images of thesubject person are captured in a case where it is difficult to determinethe state of the subject person as described later. In this embodiment,the predetermined number of times a still image is to be captured is 300to 500, for example, and the imaging time is approximately 3 to 5minutes.

The CPU 14 checks the position of the subject person, based on theoutput of the imaging element of the imaging unit 2 (step 4). This is tocorrect for the influence of an aberration of the wide-angle lens systemin accordance with the position of the subject person. In the following,the influence of an aberration of the wide-angle lens system isdescribed.

As described above, the wide-angle lens system of this embodiment isdesigned to have a view angle of 80 degrees, and a focal length of 6.188mm. Therefore, where the height of the ceiling of the living room is 2.6m, and the height of the subject person is 1.6 m, a circle(approximately 100 m²) having a radius of 5.67 m from a pointimmediately below the wide-angle lens system is the imaging area of theimaging unit 2. Accordingly, as long as the room is a standard-sizedroom, the head of the subject person can be imaged by the wide-anglelens system of this embodiment, no matter which part of the room thesubject person is in.

When the subject person is present almost immediately below thewide-angle lens system (when the angle of view is low), the image sizeof the head of the subject person is proportional to f·tan θ (f beingthe focal length). Therefore, the relationship between the height of thehead from the floor and the size (the diameter) of the image is as shownin FIG. 5B. The change in the image when the height of the head ischanged by 300 mm, from 400 mm from the floor to 100 mm from the floor,is 68 μm, which is equivalent to approximately 12 pixels. Accordingly,when the subject person is present almost immediately below thewide-angle lens system, the image determining device 1 of thisembodiment can detect a 300-mm change in the height of the head of thesubject person.

Next, a case where the subject person is in an intermediate view-angleposition (at a view angle of approximately 70 degrees) is described.FIG. 12 is a graph showing the relationship between the view angle ofthe wide-angle lens system and the distortion. As shown in the graph,the distortion becomes larger as the angle of view becomes larger. At aview angle of 70 degrees, there is influence of a distortion ofapproximately −50%. Therefore, even if the distances from the front-sidefocal point of the wide-angle lens system are the same, the image sizeat a view angle of approximately 70 degrees is almost a half of theimage size obtained when the subject person is present almostimmediately below the wide-angle lens system. However, the change in theimage size caused when the height of the head of the subject person ischanged by 300 mm at a view angle of approximately 70 degrees is almostequivalent to 6 pixels. Therefore, the image determining device 1 ofthis embodiment can detect a 300-mm change in the height of the head ofthe subject person even at a view angle of approximately 70 degrees. Inthis case, the image determining device 1 detects the head of thesubject person obliquely from above. Therefore, the size of the head maybe corrected by taking into account the horizontal-to-vertical ratio ofthe head in accordance with the anticipated angle with respect to thewide-angle lens system.

Lastly, a case where the subject person is in an outermost peripheralview-angle position (at a view angle of approximately 80 degrees) isdescribed. As shown in FIG. 12, at a view angle of approximately 80degrees, there is influence of a distortion of approximately −80%, andthe magnification is almost 20% of the magnification at the center viewangle. Further, the distance from the front-side focal point of thewide-angle lens system to the head of the subject person is longer thanthe distance observed when the subject person is present at the centerview angle. Therefore, a change in the image size in a case where theheight of the head is changed becomes smaller. Specifically, in a casewhere the subject person is present at a view angle of approximately 80degrees, the change in the image size caused when the height of the headis changed by 300 mm is almost equivalent to one pixel. Therefore, it isdifficult for the image determining unit 1 of this embodiment to detecta 300-mm change in the height of the head of the subject person at aview angle of approximately 80 degrees. However, even if the room is aliving room, an outermost peripheral view-angle position of thewide-angle lens system is a corner of the room, and there are structuralobjects each having a known size, such as a window and a door. In viewof this, images of the window and the door in corners of the room arecaptured in this embodiment, and positions of the head of the subjectperson are associated with the positions of the structural objects suchas the window and the door, so that a change in the position of the headof the subject person can be detected even in an outermost peripheralview-angle position of the wide-angle lens system.

If two or more wide-angle lens systems are provided, the accuracy ofdetection performed when the subject person is present in a corner ofthe room can be made higher, and it is possible to cope with a largerliving room.

In a case where the number of times imaging has been performed is 300 to500, which has been determined in step 3, a still image is captured instep 5, and the amount of image characteristics is calculated in step 6.Specifically, the size of the head in the Z-direction is associated withthe X- and Y-directions, and is calculated from the captured stillimage.

The CPU 14 determines whether the state of the subject person can bedetermined from the position of the subject person in the X- andY-directions and the calculated head size (step 7).

FIG. 13 is a diagram illustrating a situation where the subject personis sitting in a chair in front of a desk. As shown in FIG. 13, when thesubject person is sitting in a chair, the size of the head is not muchdifferent from that in a case where the subject person is having aproblem and leaning against the chair or is sleeping at the desk. Insuch a case, it is difficult to determine whether the subject person isin a normal state or in an abnormal state. As described above, the imagedetermining device 1 can recognize an approximately 100-mm difference inheight. However, when the anticipated angle of the wide-angle lenssystem with respect to the subject person is large, or when the subjectperson is sitting in a chair, the distance from the wide-angle lenssystem becomes longer, and therefore, step 7 is provided.

In such a case, the CPU 14 determines that a check cannot be made todetermine whether the subject person is in a normal state, and returnsto step 3. A still image is then captured several hundred times inseveral minutes as described above, and attention is paid to the head ofthe subject person as indicated by the dotted-line circles. A check isthen made to determine whether the head is moving.

If the CPU 14 determines that the state of the subject person can bedetermined after the predetermined number of imaging operations, the CPU14 determines the state of the subject person in step 8.

If the CPU 14 determines that the subject person is in a normal state,the CPU 14 stands by until a predetermined period of time (15 minutessince the start of the timing, for example) passes in step 9. Afterthat, the CPU 14 returns to step 1. Even when the subject person is in anormal state, the CPU 14 notifies the designated device that there is noproblem, at intervals (at 8-hour intervals; 7 am, 3 pm, and 11 pm, forexample) set in the setting unit 7.

If the CPU 14 determines that the subject person is in an abnormalstate, the CPU 14 moves on to step 10.

FIG. 14 shows a still image captured when the subject person isdetermined to be in an abnormal state. As shown in FIG. 14, even thoughthe subject person is in the walking area of the living room in the X-and Y-directions, the height of the head of the subject person is lowerthan Z2 and is on the floor of the living room. In this case, the sizeof the head of the subject person is much smaller than the size of thehead in the walking area, and therefore, the CPU 14 determines thatthere is something wrong with the subject person.

In step 10, the CPU 14 notifies the designated device that the subjectperson is in an abnormal state.

The CPU 14 determines whether there is a request for an image from aportable telephone or a personal computer of a family member with a lowprivacy level that is set in advance (step 11). If there is a requestfor an image, an image is transferred in step 12, and an instructionfrom the designated device is awaited (step 13). Even in transferring animage in step 12, the CPU 14 prohibits a transfer of any still imagecaptured while the subject person is determined to be in a normal state,and transfers an image captured when the subject person is determined tobe in an abnormal state. In this manner, the privacy of the subjectperson is secured.

If there is not a request for an image, the CPU 14 moves on to step 13,and awaits an instruction from the designated device. When the imagedetermining device of this embodiment transfers an image, either a stillimage or a video image can be transferred.

Meanwhile, if the CPU 14 determines, in step 3, that the number ofimaging operations performed exceeds the predetermined number, the CPU14 moves on to step 14. The CPU 14 then uses the speaker 10, to talk tothe subject person, “Are you all right?” or “How are you feeling?.” Atthis point, the CPU 14 controls the interface unit 8 to lower the volumeof the sound source device such as an audio device (or to mute theaudio).

The CPU 10 collects the voice of the subject person via the microphone 9(step 15), and uses an audio dictionary (not shown) to carry out anaudio analysis (step 16). In this case, the phrases the subject personwould use are limited to rather typical ones, such as “I'm all right,”“What?,” “I feel ill,” and “Help me.” Therefore, the audio dictionarydoes not become large (complicated), and there is no need to carry out asophisticated audio analysis such as breaking down a conversation intoindividual words.

The speaker 10 and the microphone 9 are directed toward the position ofthe subject person by a drive unit (not shown) such as a stepping motor,so that the inquiry through the speaker 10 and the voice collection bythe microphone 9 can be efficiently conducted.

If the CPU 14 recognizes a phrase indicating a normal state from thesubject person, the CPU 14 returns to step 1. If there is a phraseindicating an abnormal state from the subject person or there is noresponse from the subject person, the CPU 14 determines that the subjectperson is in an abnormal state, and moves on to step 10 (step 17).

Prior to the notification of an abnormal state in step 10, an inquirythrough the microphone 9 may be made as shown in step 14. In thismanner, a more specific notification indicating not only that thesubject person is having a problem but also that the subject person ishaving a headache or is unable to speak or the like can be sent to thedesignated device.

Modification

In the above described embodiment, the imaging unit 2 is attached to theceiling, and images of the head of a subject person are captured fromabove. However, the imaging unit 2 may be installed in a living room ora bedroom so that images of a subject person can be captured from theside. By capturing images of the subject person from the side, movementof the eyes, the mouth, the nose, and the like can be easily determined.Further, still images captured by the imaging unit 2 attached to theceiling and still images of the subject person captured from the sidecan be used at the same time in determining whether the subject personis in a normal state. In this manner, determination with higherprecision can be performed.

Also, in the above described embodiment, a check is made to determinewhether the subject person is present in a living room by thepyroelectric sensor 6. However, the intervals at which the imaging unit2 performs imaging may be shortened, and, based on the results of theimaging, a check may be made to determine whether the subject person ispresent in the living room. In this case, the pyroelectric sensor 6 maynot be provided.

In the above described embodiment, the image determining device of thisembodiment is installed in a living room or a bedroom. However, theimage determining device may of course be installed in a hallway, arestroom, a bathroom, or the like, and can also be used in an office ora school.

While the exemplary embodiments of the present invention have beenillustrated in detail, the present invention is not limited to theabove-mentioned embodiments, and other embodiments, variations andmodifications may be made without departing from the scope of thepresent invention.

1. (canceled)
 2. An imaging device comprising: an imaging sensor thatcaptures image data; and a processor configured to: acquire heightinformation about a first person and height information about a secondperson based on an object included in the image data; and identify thefirst person and the second person based on the height information aboutthe first person and the height information about the second person. 3.The imaging device of claim 2, wherein the processor determines a stateof at least one of the first person and the second person based on theheight information about the first person and the height informationabout the second person.
 4. The imaging device of claim 2, wherein theimaging sensor images the heads of the first person and the secondperson.
 5. The imaging device of claim 2, wherein the imaging sensorperforms imaging when at least one of the first person and the secondperson moves from an imaging area of the imaging sensor to a non-imagingarea of the imaging sensor.
 6. The imaging device of claim 2, whereinthe processor acquires respective heights of the first person and thesecond person as the height information.
 7. The imaging device of claim2, wherein the imaging sensor simultaneously images the first person andthe second person.
 8. The imaging device of claim 2, wherein the imagingsensor images the first person and the second person when the firstperson and the second person are in the same posture.
 9. An imagingdevice comprising: a lens; an imaging sensor that captures image databased on light through the lens; and a processor configured to: acquireheight information about a first person and height information about asecond person based on an object included in the image data; identifythe first person and the second person based on the height informationabout the first person and the height information about the secondperson; and based on the acquired height information, output anotification when at least one of the first person and the second personis in an abnormal state.
 10. The imaging device of claim 9, wherein thenotification is output by way of a speaker.